The Future of No Child Left Behind
Tuesday, April 20, 2004
by Bruce Thompson

In recent months, criticism of the No Child Left Behind act has escalated. Former governor Howard Dean made it one of the centerpieces of his campaign for the presidency, attacking his Democratic opponents for having supported it. The central provisions of the act, those requiring schools to test their students and that impose sanctions if the results don’t meet standards, have become a particular focus for attacks from politicians, state and federal legislators, educators, school boards and others.

             The act was predicated on several basic principals. First, despite the billions spent on education over the years by the federal government, it was hard to find evidence that students’ education had benefited. It was felt that the equity argument for federal funding–the notion that federal money helped balance the usual advantage of rich districts–was no longer enough. Instead there was a growing consensus that the federal government should demand results in exchange for its aid to education.

             A second was that educational efforts should be judged on their outcomes. While apparent common sense, this view was a direct challenge to the process orientation dominating American education. As a school board member, I recall my frustration when being asked to approve new texts or curriculum changes. Rather than being given evidence that students learned more under these changes, we would be told about the exhaustive deliberations of the committees and the number of constituencies consulted in the decision process.

             Another underlying principal was the belief that more measurement of results was needed. Previously, in most states, students were measured only three times in their school career. Additionally, it was hard to compare the results from one state to another, since both the tests and the standards used to report the results varied widely.

             A fourth principal was that, before widespread adoption, programs should be tested to assure they improved student learning. Too often, programs were chosen based on conformance to some educational theory, rather than any evidence of effectiveness.

             Finally, it was believed that schools should be encouraged to be concerned about all their students. By reporting average scores, a school system could appear to be doing well while one or more subgroups of students was being left behind.

             Contrary to the critics, the law is having a number of positive impacts. Under pressure to improve, school systems have become more open to promising, if unstylish, programs. Some of the barriers to communication among schools are disappearing as they attempt to learn from their more-successful peers. The law has forced student achievement to the top of the agenda for many schools, administrators, and school boards. There is more interest in learning what evidence supports claims for the success of educational programs.

             Thus it would be easy for supporters of the NCLB to dismiss the criticisms as simply signs the law is working. The protests can be seen as the predictable cries of supporters of an increasingly discredited status quo, whose favorite nostrums are found wanting. In this view, the answer is simply to hang tough until the benefits of the law become manifest.

             However, as the law’s provisions become increasingly demanding, its limitations will become increasingly evident. These limitations will cause growing resistance. Unless supporters of the law’s basic principals act now to begin to address its weaknesses, we may see a reaction that will sweep away its most important reforms.

             The law’s problems, in my view, stem in part from an underlying utopianism, best exemplified by its demand that all students reach the same level of proficiency. This concept denies the wide variety of student abilities and the influences beyond the school. It implies that schools have almost unlimited power over student achievement. Unless the requirements are set very low, in the real world some number of students will not be able to meet the proficiency level despite the schools’ best efforts. Conversely, for other students meeting the standards will be easy with minimal effort, perhaps resulting in underachieving students and overly complacent schools.

             Unless placed into some sort of context, proficiency levels are arbitrary. Proficiency for what? Defenders of the proficiency notion like to use the analogy of an airline pilot who must have certain well defined skills. The aspiring pilot cannot, for example, compensate for a failing grade in landing through a superior grade in taxiing. High school graduates, by contrast, go on to a variety of jobs or education, whose requirements vary widely. Without answering the question of proficiency for what, it is not surprising that proficiency requirements vary widely from state to state or from one grade to another within a state.

             Compounding utopianism are problems created by unresolved federalism. While the federal law requires schools to assure that a steadily increasing fraction of students are proficient it leaves the definition of proficiency up to the states, creating an incentive for states to discover that their proficiency levels were set too high.

             Most of the schools singled out for sanctions will be those serving low-income neighborhoods. The likely result is to make it even harder for those schools to recruit able teachers and principals.

             Finally, by judging schools on the percentage of students judged proficient, the law creates an incentive for schools to concentrate their efforts on a subgroup of their students, those close to the proficiency cut-off whom a bit of extra effort can push over the line. The others, either safely above the proficiency line or so far below that no amount of effort is likely to push them over, may be neglected.

             Is there a way out of this quandary? I believe there is. The answer lies in applying a bit of statistics to the test scores the law already requires.

             Starting about forty years ago a series of reports appeared describing the impact of poverty and other socio-economic factors on student achievement. Over the same period research was unable to measure significant effects on achievement from readily-available school characteristics, such as those used for accreditation. This combination of strong effects of family characteristics and weak measurable school effects led many to conclude that student achievement was mostly outside the school’s control.

             In essence, this conclusion turned the traditional relationship between education and upward mobility on its head. Rather than education as a route out of poverty, elimination of poverty became prerequisite for a good education.

             Researchers were initially interested in using the results to better understand which inputs had the most impact on education. Yet the models were far from perfect. They managed to predict only a portion–and a relatively small one–of the differences between schools. Results from schools serving apparently similar populations differed substantially.

             This implied that something the schools do significantly affects student achievement. That earlier studies missed these differences may imply that the studies were looking at the wrong characteristics. The characteristics that were measured (often measures of resources used) were not those affecting student achievement. It appears instead that the crucial differences between schools reflect differences in curricula, teachers, and leadership.

             It is this gap between predicted and actual performance (called the residual) that offers a way out of the quandary faced by the NCLB. Some schools are substantially below their predicted performance; others substantially above. These differences persist for years and may cover a variety of subjects and grades. Often they change only when a new principal takes over or when the school adopts a new program.

             Schools judged in need of improvement would be those with large negative residuals. This approach has two immediate and obvious advantages. The differences among states because of differing proficiency levels would become irrelevant. Secondly, under performing schools serving middle class students would lose their immunity from intervention.

             At the same time, schools consistently performing above prediction could be treated as laboratories. They could be studied to learn what they do differently and how their practices could be transferred to the large number of schools in the middle.

             It is likely this approach will be criticized by both sides in the education wars: those who believe that nothing schools can do will raise student achievement and those who believe the schools can do everything. Some will object to the use of income or ethnic data for fear of perpetuating low expectations for poor and minority students. They may propose that only data on test scores of incoming students be used. However, prior student achievement is of limited usefulness in rating elementary schools since testing does not normally start until students have been at the school for several years.

             Another likely criticism is that such a rating system assures that half the schools will always be below predicted levels. In the happy event that average performance increases, schools not matching that increase will find themselves steadily sinking in the ratings until they too may be marked for improvement. To me, this sounds like an ideal outcome, with steady pressure for continuous improvement and an incentive to discover and implement the practices of schools at the top.

             A valid concern is the risk that the data used to describe the students do not fully capture important differences in groups of students that consistently affect achievement. For example, there may be cultural differences in populations served that impact educational results. In analyzing the results it is important to look for these factors, but a scheme that captures them imperfectly is certainly preferable to one that ignores them entirely.

             As the attacks increase, supporters of NCLB may be tempted to hunker down and insist that no changes are needed. The likely result is slow death as the Act’s requirements are watered down. A better response is a mid-course correction building on the increased data available to move towards incentives for continuous improvement in student achievement.

Bruce Thompson is a professor at the Rader School of Business, Milwaukee School of Engineering and former president of the Milwaukee Board of School Directors. His description of an alternative means of rating schools appears in the March 2004 issue of Urban Education.